View Full Version : Milkymist - MilkDrop on FPGA
28th July 2009, 16:27
Just to let you know that I'm working on an implementation of MilkDrop-like visual synthesizer on a FPGA-based "open source hardware" system-on-chip.
The current status is: the system-on-chip design is almost done, and the rendering software is in a very early stage.
And I still have some trouble with some details of MilkDrop... for example, how is it supposed to work when dx or dy is more than 1? (from what I understand, that would shift the complete picture out of the screen, but apparently I did not understand correctly). I have to RTFS but it's really time consuming.
The project URL is http://www.milkymist.org
If you are interested in the project feel free to join the mailing list at http://lists.milkymist.org. We welcome all suggestions and contributions.
28th July 2009, 16:29
URLs: milkymist dot org, lists dot milkymist dot org
28th July 2009, 17:41
Interesting! Is it really possible to pack the power of a modern video card into an FPGA ? I suppose this does not include shader support, right ?
As regards dx and dy, they are simply wrapped, i.e. dx=0 is the same as dx=1.
dy = 0.1 means vertical shift by 10% of the image height, per frame.
Quite generally, unless specified otherwise, milkdrop would wrap all textures, which means any index outside (0;0) to (1;1) would be wrapped back to this range, so an access to (2.3; 17) would be the same as (.3; 0).
28th July 2009, 22:18
Having the power of modern video cards on FPGAs requires this type of hardware:
www. dinigroup . com/index.php?product=DN8000k10
which is obviously extremely expensive. The big video card companies use them to prototype their chips, but they are not affordable for most people.
Milkymist works on much smaller (and cheaper) FPGA boards. MilkDrop does not require that much power, and parts of the DirectX/OpenGL pipeline that are not used can be removed (or implemented in software if they do not need a lot of power - drawing waves for example only uses about 10-15% of the CPU time of my softcore).
There is currently no pixel shader support, even though it is certainly feasible even on a small FPGA.
Thanks for the explanations about the wrapping. I will be delighted to implement floating point modulo in the FPU *grin*
Is it the same for cx/cy?
29th July 2009, 19:32
Son of Monster FPGA. Cool. 22.000 USD. Bit much really.
cx/cy is still a bloody mystery to me. With respect to rotation, the default seems to be (cx,cy) = (.5,.5) which results in a rotation around the centre of the screen. That would be reasonable, and is according to spec which says that cx/cy determine the center of motion for rot and stretch.
With respect to zoom, cx does not seem to have an effect, except however for large values; cx = 50 will move the image slowly to the right... probably an artefact.
29th July 2009, 22:21
22000 USD would be cheap. It's five times that.
From the source code, cx/cy is used for the zoom (defines the point on which to center the zoom), scaling (sx/sy - that I do not support yet) and rotation.
Yes, I also think large values of cx should not move the image. That's probably a problem with floating point precision.
What I don't understand currently is how the texture is wrapped (which is probably related to the behavior when dx>1). It seems that MilkDrop sets the wrapping on the *texture* coordinates (the projectM implementation uses glTexParameter with GL_REPEAT on GL_TEXTURE_WRAP_S + GL_TEXTURE_WRAP_T), but said texture coordinates form a simple rectangular mesh that is hardcoded (does not depend on the preset), and are always between 0 and 1 (and therefore are never wrapped). It would be more coherent if the *vertex* cooordinates were wrapped, but then you have problems with off-screen vertices appearing because of zoom or rotation.
30th July 2009, 06:01
bonjour sÃ©bastien ;)
i have noticed your work by your thread on the sourceforge projectm forum and i was just about to start a thread on milkymist here myself ;) interesting work!
see this thread:
it's taking a closer look at how the mesh works and i guess it's kinda interesting for you.
and you might also like the wiimote hack:
by milkdropesque do you mean it can interpret original milkdrop presets, or at least a subset?
30th July 2009, 12:38
Yes, the target is to implement as many MilkDrop features as possible. Having the equivalent of MilkDrop 1.04 seems reasonable; supporting the latest presets with the pixel shaders need a lot more hard work (and time).
The Wiimote hack is nice stuff and is close to what I want to do: integrate MIDI, DMX512 and possibly other interfaces on the FPGA board, that will control more preset variables. And also enable the preset to control DMX, so that it can affect stage lighting in real time.
I'm also thinking about adding two PAL/NTSC video inputs, that can be integrated in the visuals in many different ways (video overlay, drawn to the texture, etc.)
Thanks for the link about the mesh thing. I will also do some experiments (like comparisons of outputs at different stages of the processing) with the projectM code to make sure I get it right.
31st July 2009, 10:08
Ah crap - I just found out I reversed the texture and vertex coordinates. Contrary to what I thought, in MilkDrop, the preset affects the texture coordinates, not the vertex coordinates. Now the "texture wrap" feature and the behavior when dx|dy>1 make perfect sense.
vBulletin® v3.8.6, Copyright ©2000-2013, Jelsoft Enterprises Ltd.