Journal of Sonic Studies, volume 6, nr. 1 (January 2014)Daniel Hug; Moritz Kemper: FROM FOLEY TO FUNCTION: A PEDAGOGICAL APPROACH TO SOUND DESIGN FOR NOVEL INTERACTIONS
The Emergence of Sonic Interaction Design Pedagogics
The areas of knowledge described above provide the basis of a systematic approach to designing sounds for interactive applications. But design education also relies on experience and practical applications, for examples and learning material, and these are scarce to non-existing outside academic experimentations. There is a need to involve the future designers in the process of exploring and defining the opportunities, challenges, and best practice in terms of sound design for interactive commodities, because they will be the ones who contribute significantly to the standard-setting postulations we discussed above. This leads to the question as to how sound can be integrated into interaction design education.
Luckily, we do not have to start from scratch. There is an abundance of textbooks available, in particular in the area of film sound and music or general sound design for media (Sonnenschein 2001; Raffaseder 2002; Viers 2008; Katz 2007; Rose 2008). In relation to interactive media, there is a smaller, but increasing, amount of material available. Many educational books focus on the technical aspects of interactive sound production (Cook 2002; Cancellaro 2006; Farnell 2010), and some focus specifically on audio for computer games, which in some way are the “killer apps” of interactive sound (Marks 2001; Collins 2008). In media and design education, we can observe that sound is becoming increasingly established, with several educational institutions offering bachelor or master degrees in sound design (for media, or in general), or at least devoting a reasonable amount of their curriculum to sound. However, educational programs specifically directed toward interactive sound design are rather rare.
Yet, over the last five to ten years, and in particular in the context of the COST action on Sonic Interaction Design, several proposals regarding pedagogical approaches to sound design for interactive applications have been made and also implemented in workshops and courses. Rocchesso, Serafin and Rinott (Rocchesso, Serafin and Rinott 2013) provide an overview over more recent pedagogical efforts in the area of SID. These efforts are directed on the one hand towards sensitizing participants to sonic interactions, for instance through sound walks or through the production of audio dramas. On the other hand, they are directed at sketching and prototyping with particular toolsets, for instance using vocal sketching (Ekman and Rinott 2010) and sonic overlay of video, but also more complex tools such as the Sound Design Toolkit (Delle Monache, Polotti and Rocchesso 2010).
Franinovic, Hug and Visell (Franinovic, Hug and Visell 2007) have presented conceptual considerations and their implementation in a workshop setting that emphasizes the exploration of basic action-sound relationships. Through methods such as morphological analysis and bodystorming, they encouraged the development of concepts, which were then explored in short video scenarios or through building functional prototypes. An exploration of basic sound design methods in pedagogics, which are inspired by methods of the Bauhaus, can be found for instance in Franinovic and Visell (Franinovic and Visell 2008).
Other efforts focus more on hermeneutics, interpretational phenomena, and dialogical communication processes. Hug has conducted a moderated group analysis of a systematic collection of filmic representations of interaction scenarios and derived “narrative metatopics” as conceptual and formal building elements for sonic interaction narratives (Hug 2010a).
Hug (Hug 2010b) also discussed the need to enable real-time improvisation of sounds in order to tackle the genuinely performative nature of sonic interaction, which we have described above. This conceptual approach has certain implications in design pedagogics, which are also of central importance in the work presented here.
In some cases the pedagogical efforts are connected to practice-based research. Hug has described a way to establish sound in a game design bachelor education, not only as topic of teaching, but also as classroom-driven research (Hug 2007). Core elements of this approach consist in the training of listening, an understanding of functions of sound in computer games, and a structured design process of prototyping and implementation of sound in interactive games. An important aspect is the continuous encouragement of experimental and unconventional approaches, which can lead to new insights about possibilities for interactive (game) sound, an approach which is also adopted in the work described here.
Last, but not least, software tools and technical frameworks are important elements in Sonic Interaction Design education as they offer the necessary functionality for realizing interactively controllable sounds. For instance, Delle Monache et al. (Delle Monache et al. 2010) have presented a software toolkit that provides a relatively straightforward way to use procedural, physics-inspired sound synthesis methods for real time applications. Other examples are the “Musical Modules” and the Gesture Follower toolkit (Schnell, Bevilacqua, Rasamimana, Blois, Guedy and Flety 2011).