Computer music with graphical inputs
February 16, 2005 9:22 PM   Subscribe

Computer music with graphical inputs:
My roommate is doing a research project on experimental/computer music where the input is some sort of visual or graphical something and the output is sound. Help him out by sharing your best links. [+]
posted by mai to Media & Arts (20 answers total)
One example is if the user could draw a line on the screen, and depending on location and speed of drawing, different notes or sounds would come out. Have you seen anything like this that rocked or sucked? Please share.
posted by mai at 9:24 PM on February 16, 2005

Oh crap. Those tags were supposed to be one. I'm sorry.
posted by mai at 9:25 PM on February 16, 2005

This is one of my favorites (Flash required).
posted by adamkempa at 9:32 PM on February 16, 2005

Fruity Loops (or FL Studio, or whatever they're calling it these days) has an image-map synth, where you load in a gif or jpg and it produces sound based on the colour distribution in the image.
posted by pompomtom at 9:39 PM on February 16, 2005

Check out:


...and ...

MIT's Hyperscore
posted by ericb at 9:49 PM on February 16, 2005

Try the various "Dub Selecta" apps over at Infinite Wheel.
posted by Jimbob at 9:54 PM on February 16, 2005

These are cool. Thanks everyone.

Also interested in stuff where it's not just that the interface is visually interesting but that the sounds are somehow related to the image. The FL color map thing sounds like a pretty great idea, but I couldn't find it on the site.
posted by mai at 9:57 PM on February 16, 2005

John Jannone (who was on the faculty at RPI while I was there) used to use a WACOM tablet (and MAX) to manipulate sound.
posted by stefnet at 9:59 PM on February 16, 2005

Processing 1.0. It is a programming language/environment whose sole reason for existing is to do this very thing.
posted by basicchannel at 10:15 PM on February 16, 2005

This is probably at the extreme end of anything you considered:

Fluid for the Playstation 1. (Try also Mobygames' page, and this link -- can't find anything better ATM)
posted by krisjohn at 11:01 PM on February 16, 2005

There was a program for the Mac in the late 80's/early 90's that actually did algorithmic composition based on mouse positioning... I'm beating my brain but can't remember the name.

Also, there was a Voice of the Goat Cactus, which I actually found via MeFi textad about a year ago. Very odd program. I never figured out exactly what to do with it.
posted by weston at 12:03 AM on February 17, 2005

Metasynth is a program that allows you use simple Paint-type tools to draw on a surface. Then the program will go left-right and make it into sounds based on the color and XY position.

Soundhack is an experimental music tool that lets you input any image in and make sounds out of it. Actually Metasynth can do that too, but Soundhack is freeware and linked off of University websites it's so cool.

I used to play with Soundhack quite a bit. You can also reverse it and make sound into images/ movies using Soundhack. It's... colorful.

You can also find a spectrogram program that will give you a bit better/more tweakable image output.

Aphex Twin very famously did the same on one of the Windowlicker EP tracks "Complex Equation" (not called that, it is one). If you set a spectrogram up right you get an image of Aphex's face just near the end of the song. Or course it just sounds like static.
posted by Napierzaza at 4:17 AM on February 17, 2005

JazzMutant's Lemur is really cool (and really expensive, at least for my budget).

I have played with Sonasphere a bit -- lots of fun.
posted by Acetylene at 6:34 AM on February 17, 2005

This isn't exactly what you're asking about, but somewhat related, and cool nonetheless!

Electronic god Aphex Twin has a song on the EP Windowlicker that, when run through a spectrograph (a program that analyzes sound waves and visualizes them), a "demon face" appears.

Pretty cool stuff. Here's a page detailing the process and showing some examples of it.
posted by nitsuj at 7:08 AM on February 17, 2005

More research yields: Coagula.

Coagula - Industrial Strength Color-Note Organ is a tool for importing, creating and converting .bmp image files to .wav sound files. The program reads the image data as sine waves and plays back the file after the conversion. A given line within the image controls the amplitude of one oscillator at a specific pitch. Vertical position is translated as frequency while horizontal position is interpreted along the time line. Red/left and green/right correspond to control stereo placement. Brighter colors produce louder sounds. Areas of black produce silence. It is possible to set the length and frequency range prior to the conversion (ie: a 20 min. audio file limited to 500.00 to 120.00 Hz).
posted by nitsuj at 7:12 AM on February 17, 2005

One guy who's always doing stuff with visual and gesture-controlled audio is Golan Levin.

Check out Manual Input Sessions (2004) and Loom (1999). His Messa di Voce performance goes the other way, letting the audience see visualizations of vocalizations made by 2 performers.

If you email Golan, he might be able to share working toys with you (mostly java apps).
posted by zpousman at 7:17 AM on February 17, 2005

Steim makes an application for Mac OS9 called BigEye. Video input (from a camera or a movie file) generates MIDI data, which is sent to the synth/sampler/MIDI-enabled thingamajig of your choice.
posted by Dean King at 8:34 AM on February 17, 2005

Balldroppings! It's hotness.
posted by 31d1 at 8:47 AM on February 17, 2005

It's not quite the same, but Mousing is a mouse-movement-controlled theremin synthesizer that reacts pretty well to artistic-type mouse squigglings.
posted by Drexen at 9:23 AM on February 17, 2005

AudioMulch can do some of these things with its metasurface. i dont know how to do it, but audiomulch is a really powerful tool.
posted by yeahyeahyeahwhoo at 12:12 PM on February 17, 2005

« Older Looking for community blogs   |   What's the hype around Moleskine Newer »
This thread is closed to new comments.