Bodily Movement and the Music Theory Paradigm: The University of Oslo Department of Musicology is developing the Musical Gestures Toolbox, a collection of applications with useful features for video and audio analysis. The toolbox is built with Max / MSP / Jitter from Cycling ‘74, and will be available as standalone applications for Mac OS X and Windows, as well as the original patches.

It features: (1) Video capturing from USB/FW-cameras (2) Playback of video files (any QuickTime-readable format) (3) Adjustable resolution and playback rate (4) Brightness, contrast, saturation controls (5) Zoom, offset and displacement controls (6) Cropping functions (drag and crop) (7) Automatic cropping based on contraction (8) Computer vision analysis (9) Save snapshots and image sequences (10) Save video from the analysis (11) Contraction and centre of gravity (12) Audio analysis: spectrogram, sonogram, spectral centroid, noisiness, loudness…

“…(T)here are many different kinds of gestures associated with music, but it could be useful to consider gestures in view of the following three main categories:

A. Sound-producing gestures, such as hitting, stroking, bowing, blowing, singing, kicking, etc. Mental images of such gestures, including the associated modes of execution such as fast, slow, hard, soft, short, long, etc., are usually indissociable from our notions of musical sound, evident in music-related metaphors (e.g. “hammering”, “sweeping”, “caressing”, etc.) and mimicry (e.g. playing “air drums” or “air guitar”).

B. Sound-accompanying gestures, including all kinds of movements we can make to music such as marching, dancing, and more vague sound-tracing gestures such as following the melodic contours, rhythmical / textural patterns, timbral or dynamical evolutions, etc. with our hands, arms, torso, etc.

C. Amodal, affective or emotive gestures, including all the movements and/or mental images of movements associated with more global sensations of the music, such as images of effort, velocity, impatience, unrest, calm, balance, elation, anger, etc., gestural images and concepts which are also encountered in dance.

These categories of gestures often overlap, and a gesture may belong to more than one category, e.g. energetic drumming may be perceived both as sound-producing gestures and as emotive images of joy or elation. Musical gestures thus encompass a large territory stretching from details of sound-production to more global emotive and aesthetic images of music, and also include considerations of cultural-stylistic vs. more universal modes of expression. In all cases, we believe musical gestures manifest the primordial role of human movement in music. For this reason, we speak of embodied perception and cognition in music in the sense that we as listeners relate musical sound to mental images of gestures, i.e. that listening (or even merely imagining music) also is a process of incessant mental re-enactment of musical gestures.

We believe the idea of embodied perception and cognition could represent a change of paradigm in music theory and other music related research, research which has often tended to exclude considerations of bodily movement from its conceptual apparatus in favour of focus on more abstract, notation-based elements of music. In our project, the focus on musical gestures provides us with a coherent and unifying perspective for what we see as a much needed renewal of music theory and other music research. Fortunately, recent developments within the cognitive sciences, music technology, and technologies for capturing and representing gestural data, converge to give us very favourable circumstances for this shift of focus towards musical gestures.”