Old 28th July 2009, 16:27   #1
lekernel
Junior Member
 
Join Date: Jul 2009
Posts: 9
Milkymist - MilkDrop on FPGA

Hi,

Just to let you know that I'm working on an implementation of MilkDrop-like visual synthesizer on a FPGA-based "open source hardware" system-on-chip.

The current status is: the system-on-chip design is almost done, and the rendering software is in a very early stage.

And I still have some trouble with some details of MilkDrop... for example, how is it supposed to work when dx or dy is more than 1? (from what I understand, that would shift the complete picture out of the screen, but apparently I did not understand correctly). I have to RTFS but it's really time consuming.

The project URL is http://www.milkymist.org

If you are interested in the project feel free to join the mailing list at http://lists.milkymist.org. We welcome all suggestions and contributions.

Best regards,
Sébastien
lekernel is offline   Reply With Quote
Old 28th July 2009, 16:29   #2
lekernel
Junior Member
 
Join Date: Jul 2009
Posts: 9
URLs: milkymist dot org, lists dot milkymist dot org
lekernel is offline   Reply With Quote
Old 28th July 2009, 17:41   #3
Nitorami
Major Dude
 
Join Date: Mar 2008
Location: Erlangen
Posts: 872
Interesting! Is it really possible to pack the power of a modern video card into an FPGA ? I suppose this does not include shader support, right ?

As regards dx and dy, they are simply wrapped, i.e. dx=0 is the same as dx=1.
dy = 0.1 means vertical shift by 10% of the image height, per frame.

Quite generally, unless specified otherwise, milkdrop would wrap all textures, which means any index outside (0;0) to (1;1) would be wrapped back to this range, so an access to (2.3; 17) would be the same as (.3; 0).
Nitorami is offline   Reply With Quote
Old 28th July 2009, 22:18   #4
lekernel
Junior Member
 
Join Date: Jul 2009
Posts: 9
Having the power of modern video cards on FPGAs requires this type of hardware:
www. dinigroup . com/index.php?product=DN8000k10
which is obviously extremely expensive. The big video card companies use them to prototype their chips, but they are not affordable for most people.

Milkymist works on much smaller (and cheaper) FPGA boards. MilkDrop does not require that much power, and parts of the DirectX/OpenGL pipeline that are not used can be removed (or implemented in software if they do not need a lot of power - drawing waves for example only uses about 10-15% of the CPU time of my softcore).

There is currently no pixel shader support, even though it is certainly feasible even on a small FPGA.

Thanks for the explanations about the wrapping. I will be delighted to implement floating point modulo in the FPU *grin*

Is it the same for cx/cy?
lekernel is offline   Reply With Quote
Old 29th July 2009, 19:32   #5
Nitorami
Major Dude
 
Join Date: Mar 2008
Location: Erlangen
Posts: 872
Son of Monster FPGA. Cool. 22.000 USD. Bit much really.

cx/cy is still a bloody mystery to me. With respect to rotation, the default seems to be (cx,cy) = (.5,.5) which results in a rotation around the centre of the screen. That would be reasonable, and is according to spec which says that cx/cy determine the center of motion for rot and stretch.

With respect to zoom, cx does not seem to have an effect, except however for large values; cx = 50 will move the image slowly to the right... probably an artefact.
Nitorami is offline   Reply With Quote
Old 29th July 2009, 22:21   #6
lekernel
Junior Member
 
Join Date: Jul 2009
Posts: 9
22000 USD would be cheap. It's five times that.

From the source code, cx/cy is used for the zoom (defines the point on which to center the zoom), scaling (sx/sy - that I do not support yet) and rotation.

Yes, I also think large values of cx should not move the image. That's probably a problem with floating point precision.

What I don't understand currently is how the texture is wrapped (which is probably related to the behavior when dx>1). It seems that MilkDrop sets the wrapping on the *texture* coordinates (the projectM implementation uses glTexParameter with GL_REPEAT on GL_TEXTURE_WRAP_S + GL_TEXTURE_WRAP_T), but said texture coordinates form a simple rectangular mesh that is hardcoded (does not depend on the preset), and are always between 0 and 1 (and therefore are never wrapped). It would be more coherent if the *vertex* cooordinates were wrapped, but then you have problems with off-screen vertices appearing because of zoom or rotation.
lekernel is offline   Reply With Quote
Old 30th July 2009, 06:01   #7
Flexi
wellspring of milk
Major Dude
 
Flexi's Avatar
 
Join Date: Apr 2007
Location: 54.089866,12.11168,18.75
Posts: 2,058
Send a message via ICQ to Flexi
bonjour sébastien
i have noticed your work by your thread on the sourceforge projectm forum and i was just about to start a thread on milkymist here myself interesting work!

see this thread:
http://forums.winamp.com/showthread....hreadid=276538
it's taking a closer look at how the mesh works and i guess it's kinda interesting for you.

and you might also like the wiimote hack:
http://forums.winamp.com/showthread....hreadid=289007

by milkdropesque do you mean it can interpret original milkdrop presets, or at least a subset?
Flexi is offline   Reply With Quote
Old 30th July 2009, 12:38   #8
lekernel
Junior Member
 
Join Date: Jul 2009
Posts: 9
Yes, the target is to implement as many MilkDrop features as possible. Having the equivalent of MilkDrop 1.04 seems reasonable; supporting the latest presets with the pixel shaders need a lot more hard work (and time).

The Wiimote hack is nice stuff and is close to what I want to do: integrate MIDI, DMX512 and possibly other interfaces on the FPGA board, that will control more preset variables. And also enable the preset to control DMX, so that it can affect stage lighting in real time.

I'm also thinking about adding two PAL/NTSC video inputs, that can be integrated in the visuals in many different ways (video overlay, drawn to the texture, etc.)

Thanks for the link about the mesh thing. I will also do some experiments (like comparisons of outputs at different stages of the processing) with the projectM code to make sure I get it right.
lekernel is offline   Reply With Quote
Old 31st July 2009, 10:08   #9
lekernel
Junior Member
 
Join Date: Jul 2009
Posts: 9
Ah crap - I just found out I reversed the texture and vertex coordinates. Contrary to what I thought, in MilkDrop, the preset affects the texture coordinates, not the vertex coordinates. Now the "texture wrap" feature and the behavior when dx|dy>1 make perfect sense.
lekernel is offline   Reply With Quote
Reply
Go Back   Winamp & Shoutcast Forums > Visualizations > MilkDrop > MilkDrop Development

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Forum Jump