Video enabled audio sequencer

This is a video-audio sequencer built using Processing. The application uses the video camera to take average colour readings, while the Minim audio library is used to create synthesised sounds. The interface for this program is modeled on a simple eight step sequencer. Each row represents a sound composed via the UGen framework, and clicking on a pad activates that sound in the sequence. The colours that appear on each pad are the average colours captured via the video source. The timbre of the sounds can be changed during playback by placing different colours and saturations in front of the video camera. This manipulates various variables that affect the speed of the oscillators, or the values of the ASDR envelope to create constantly changing unique sounds.

This software study reinterprets the functionality of an eight step, six channel sequencer. A video camera is used to observe objects placed in view, with its colour averaged. The filter and effects on each channel are adjusted using colour, allowing the timbre of the generative sound to be altered.

The software presents a grid of six rows and eight columns of clickable pads, with each row representing a single instrument.

There are a two major parts to this program: audio synthesis and video image analysis. For the audio synthesis I use the new UGen framework found in the Minim audio library. This class provides a range of sound generators, envelopes and effects which can be used to create instruments. All of the sounds in this program start with a simple square or sine wave, filters and envelopes are then manipulated by the live video. The video image analysis part of this program captures video from a camera, analysed with the resulting average hue, saturation and brightness information used to manipulate the audio.