the soundrawing project marked the beginning of my computer vision research. the pitch, volume, and brightness of the sound is analyzed and used to determine the respective color, width, and height of the paintbrush. the xy location is determined by the location of the mouth.
soundrawing test from Brian McGaw on Vimeo.
the sound analysis is performed using the wonderful analyzer~ object by Tristen Jehan in max/msp:
http://web.media.mit.edu/~tristan/maxmsp.html
it’s then bussed to a processing sketch via OSC where the head-tracking is performed and then a visual representation of the sound is drawn over the mouth. there are several different key commands:
s – save image
d – stop drawing the image
c – clear the shapes
r – switch between rectangles / ellipses
l – “line mode”
o – show outline
space – save movie / close ( use this instead of the escape key )
feel free to download the application and its’ source code and post any questions you might have in the comments. if you use it to make a video or extend it, please share your work back with me as well, i’d love to see what you come up with.