Wiimote-reactive Milkdrop visuals

[ryan]

Wiitards.

[Flexi]

setup successfully completed!

Biiig Thanks for bringing me to this point, it's like a litte dream come true.
Now let's hack some stunning presets!

(@ryan: )

[Flexi]

oh i really forgot about the poor image quality of Milkdrop 1.05
(i want pixelshader )

i think, i need the acceleration values inclusive gravity. i can do the smoothing and roll pitch calculation in milkdrop code as well.
and may it be possible, that the number of variables shrunk? i just tried to remix an old preset, which already used the maximum number. (but hey - i remember, this number was raised with the 1.05beta )

here are first results. this is fun

edit:
it's probably not the best preset for dancing action.i tried a really soft dampening - inclusive overshooting - so all peaks are filtered out, but if you're lucky you get into resonance with the loop controller. you might edit the first three vars...)

[Flexi]

man this is insane...
...press A

this preset obviously uses the maximum amount of variables. it's only 12 of em? thats far to few for most presets...

[indaVizz]

this is like you're navigating a spaceship through a field of acid shapes. the colors are incredible.

here's a remix that includes beat reaction: the right wiimote paints shapes, the stronger the move the bigger the shapes, while the left wiimote navigates the "spaceship" as before, plus any force applied to it makes it move faster. (i.e. zoom depends on wii_l_f). left and right buttons are for two speeds (couldn't think of anything better).

youtube.com/watch?v=u78m9TfL9l4

[Flexi]

still practicing.
this is the code i would rather see in the per-frame section.

enjoy!

[indaVizz]

the physics of it is awesome. it feels like you're moving a live worm with the wiimotes.

can you add more direct feedback to it but still keep the lifelikeness of it? e.g. you squish it with one wiimote acceleration and stretch with another, but it's standing still when you're not moving the wiimotes? (almost still, it would be good that it still wiggles a bit but very little.) the roll and pitch of one wiimote could control its angle or how close the two ends are.

what i'm thinking is, it would be cool to make the feedback so that you always know what action your arm movement is going to make (like you do with your real arms and hands).

[indaVizz]

another wii-only mix

here's a wii remix of Rovastar's Eye on Reality mix. i just copied what redi jedi did with the wisps wii remix: left wiimote roll controls rotation, pitch controls zoom, right wiimote force (acceleration) controls the size and colors of the eye, and left wiimote force does something that i don't know what it is but looks really cool. buttons supress roll/pitch->rot/zoom.

it has great feedback (the only thing i talk about) but i think the colors and shapes are great too (that came with the package). hope you like it!

davor

p.s. i'll base the new release off of latest MD beta; let me know if you think some other vars should be supported. (i was planning to add wii_{1..7}_* values for up-to-7 wiimote support.) as for gravity-including x, y, and z acceleration, you can derive them from wii_{l,r}_{x,y,z} and roll/pitch:

wii_l_gx = wii_l_x + sin( wii_l_r );
wii_l_gy = wii_l_y - sin( wii_l_p );
wii_l_gz = wii_l_z + cos( wii_l_p );

and same thing for wii_r_*. the gx, gy, gz are the acceleration values as measured by the wiimote directly and they are most accurate. roll and pitch is calculated from there (and not quite accurate when you move the wiimotes too fast), and so are x, y, z gravity-less accelerations.

[Flexi]

no way, that roll and pitch values have most accuracy jet.
see how the the other value freaks out, when one axis is rotated by 90°. place your wiimote on a table and roll it from left to right.
(red is pitch and blue is roll)
can't you pass through the acceleration values right before gravity is subtracted, or am i totally off the mark?

and another thing:
what an effort would it be to give me input from the IR sensor? i read "Wiimote detects and transfers up to four IR hotspots" (just imagine, all i need for headtracking is two points)

[redi jedi]

Gotta check these out when I get home... damn this is so cool.

hey is there any way to get the litebar supported so we can get exact positional data, and or mabye a point that the user is point the mote at? that could be fun..

hey indaVizz, are you any good with c#? I don't care much for the fact that I dont have the V2 source(maybe saying this will help with that) so I've decided to rebuild it in C#.. mainly cause my new job is c#/vb6 based so I need the practice.. but once i get the md features implemented I'm prolly gonna switch the preset format(xml based most likely) or at least extend it alot... can you do the same thing in c#?!?

btw.. using Lua as the scripting engin(current md format executes fine) so the extendability is gonna be wicked(built in strings/arrays/classes anyone?)

[Flexi]

i have just been to our electronics store and buyed some IR-LEDs and a button cell.
Who's faster - me organizing a soldering gun or you implementing?

this will get into serious performance art!

@jedi redi: nice to hear. I have quite a lot experience with Java and need C# practice as well. Say if you need a helping hand.

[redi jedi]

helping hands are always good..

I'm thinking that at least up to being md2 feature comparable(I don't expect to match the speed) it will be open source, then I might break it off into another project(leaving the source still available for others of course) for further development but I may start totally over at that point to, I'm not sure(nor do i care right now) cause I have some other ideas I want to explore after gaining the experience of this.

But anyway I'm trying to make it fully OOP so I can reuse parts of it in other projects, right now I'm mostly doing the design parts, are you any good at class design?

@indaVizz: btw thanks for reintegrating it into my beta, thats cool.. Few requests for the next version:
if you have any more variables from the controller, expose them.

put the bass,mid,and treb back, or add a var we can set that say to turn them off, or a menu option, something..

up the min eval user variables(think its in one of the header files), are you adding to that when you add your variables? as its just a total, not the avail custom variables. its needs to be 30 over all the built in ones for md1.04 and 40-45+ for betas.

[Flexi]

[ot]
I'm studying computer science major in software-technics UML is my milk (or should be ), but i must admit i have not worked too much with it yet. I'm also quite proficient in XML (but again more in an academic manner).
Re-engineering and model-based software engineering are exactly my topics.

this is curious: I'm still waiting for my processor for my new state-of-the-art PC and i also resolved to finally switch to Vista and going into demo-stuff like wiimotes and DX10.
I'm a little delayed with my studies, but this semester i don't have very much courses... (but therefore more examinations )
[/ot]

[redi jedi]

Well heres the main design questions as I see them(remember this inst exactly suppose to be a direct copy of MD2, just have AT LEAST the features in it):

do you go with a fixed or semi-fixed render loop(like MD, this is what allows transitions to work the way they do)

or do you go with a preset driven render(presets make render calls directly) loop(possible with Lua) to give more control but resolve to bitmap based scripted transitions? then write a milkdrop translator to fill a template with the code from an md file...


I personally like transitions so I'm prone to go with a semi-fixed(kinda like the beta where you have some predetermined control over the loop, but its essentially static) loop, any thoughts on that?

Lua presents the ability to add some really cool shit as far as presets are concerned.. for instance Events, you could create an Event handler in C# that would call functions that have been registered from Lua, With a built in beat detection system you could register some code to run On Beat, maybe some other code to run OnMouseMove... that could be cool..
anyway do ya add things like events or maybe custom renders(humm.. fixed render line with scriptable renders on top? is that feaseable..)

also do you build a beat detection system into the main shell, since it will be faster there, and most real-time BD algos need some "warm up" time, plus a standard BD output will make remixing presets easier, and if they have standard out put you could switch between them aswell, mabye let the user pick the one they think works the best

[indaVizz]

new features

hey guys, thanks for the feedback, first of all!

based on the last few messages here's the list of new features i plan to add:

1. based on redi jedi's latest beta.

2. IR support -- that's a great idea! i was obsessed with the wii-wristbands (with camera soldered out) and totally forgot about it. more about IR in the next message.

3. more wii vars -- though the only ones missing at the moment are wii_*_g{x,y,z}, which are raw accelerator values. (flexi: more on that in the next message), and also wii_*_y for "yaw", the angle at which the wiimote tils in (its) horizontal plane; that one works only when IR is supported.

4. a preset-writable var to determine whether bass/mid/treb come from sound or from wii.

5. bumping up user eval vars -- i didn't know such thing existed, thanks. anyone has a preset where this number is exceeded (or i can make it easy to exceed it) so i can test?

6. supporting up to 7 wiimotes at the time: wii_{1,2,3,4,5,6,7}_* (in addition to wii_l_* and wii_r_*) for collective performance. that should be pretty straightforward, although tedious, to add.

btw could adding 100+ new input vars make running presets slow? i would guess/hope not, but i don't know how MD really works.

7. add an option that if no wiimotes are found, behaves like a regular milkdrop. (also when you press certain key combos you can turn off wii-reactivity and put back regular sound-reactivity. then if you play your music through winamp you can switch from user-driven to music-driven visuals.)

8. make left wiimote's cursor buttons decide what the right wiimote's cursor buttons do, e.g. if the last left cursor button pressed was "up", then right cursor buttons control sensitivity; if the last left button was "down" then the right cursor buttons do something else, like send their values to the preset. (it's a terrible user interface but in a performance you may need to do as much as possible without going back to the PC).


let me know if something is missing or you think is unnecessary. if you want IR support right now, i could do that first and then switch to beta.

i'll also release the source soon, the cleanup is almost done. all the changes in the code are marked with [wii], and there's about 40 small blocks of code added to the MD source and just one block commented out, plus there's a couple of wii-only files. so i'm hoping that the transition to the beta should be fairly smooth.