Raster scanning: A new approach to image sonification, sound visualization, sound analysis, and synthesis.

다국어 초록 (Multilingual Abstract)

Raster scanning is a technique for generating or recording a video image by means of a line-by-line sweep, tantamount to a data mapping scheme between one and two dimensional spaces. While this geometric structure has been widely used on many data transmission and storage systems as well as most video displaying and capturing devices, its application to audio related research or art is rare.
In this thesis, a data mapping mechanism of raster scanning is proposed as a framework for both image sonification and sound visualization. While this mechanism is simple, it has a strong potential as a cross modal representation and produces compelling results when used for sonifying image texture and visualizing sound timbre.
Special attention is paid to the rastrogram (raster visualization of sound) as an intuitive visual interface to audio data. In addition to being an efficient means of sound representation that provides meaningful display of significant auditory features, the rastrogram is applied to the area of music analysis by visualizing tempo and structure information.
Finally, a multimedia environment for interactive raster-scanned image sonification is presented. This software allows for real-time sound synthesis from images based on the method combining scanning variables with image processing filter, and thus enables complementary and analogous integration of image and sound. Applications include diagnostic exploration of visual data, creative sound design and composition, and multimedia performance.