The Journal of Sonic Studies
Journal of Sonic Studies, volume 6, nr. 1 (January 2014)Daniel Hug; Moritz Kemper: FROM FOLEY TO FUNCTION: A PEDAGOGICAL APPROACH TO SOUND DESIGN FOR NOVEL INTERACTIONS

To refer to this article use this url: http://journal.sonicstudies.org/vol06/nr01/a03

Recommendations Related to Specific Stages

Establishing Foley as Demonstration Method

next section

In order for the students to be able to use Foley for the design process required practice and minimal skills with the microphone. Otherwise, this stage could lead to an oversimplification of sounds, which leads to an unsatisfying quality of the demonstration, which in turn leads to abandoning an otherwise good approach. Live sound making as means of sonic interaction design needs to be taken seriously, and it is important to push participants to really explore the vast universe of sonic possibilities provided by objects.

Multisampling as Mediating Technology

With multisampling and MIDI mapping, this stage provides the basis where students are able to prototype their ideas in a more structured and controlled way than with the Foley method. Therefore, this stage requires a clear concept and, ideally, some existing sounds from the Foley stage to be effective. In the worse case, this leads to a rigid scripting of the interaction process and an emphasis on sequential triggering, in the positive case, the improved control over the sound production increases the expressive and narrative potential.

Most importantly, the electroacoustic live-mockup bridges the gap between Foley exploration and functional prototype, as it delineates in advance the relevant parts for the implementation as functional prototype, in particular concerning systematization of the concept and mapping of sound parameters to interactions and processes. The restriction on working with samples is useful for our target group, interaction design students, as it is consistent with the Foley stage. But in principle, the multisampler stage can be extended with other methods of digital sound generation.

Sensor Technology as Boundary

A prominent insight from the cases is the impact of the various technologies on the respective development of the projects. It is remarkable that in particular the difference between continuous and discrete acquisition of sensor data leads the students to either explore possibilities more openly, or to make restrictive decisions in a “yes” or “no” manner. The Dipalu case, for example, is based on a microphone as relatively imprecise input device, and thus motivated the students to think about situations as ambivalent and continuously changing. This ongoing exploration, coping with the technology and using it as malleable material, also contributed to the development of their concept. The opposite example can be found in that of the camera used for the realization of Sonotag. The existing functionality of the Microsoft Kinect camera was used to trigger events directly, and therefore limited the range of possible interactions. The group became more oriented towards functional implementation, which resulted in a much more defined and inflexible prototype. Also, the danger of the tracking breaking down motivated the use of a stereotypical and prominent warning sound which broke the narrative coherence.

Another point observed was that as soon as sensor technology is considered, be it conceptually or in actual implementation, it will take time and energy away from a sound-driven exploration. We strongly recommend starting with open methods for both sound generation and sensor technology and to maintain this openness as long as possible. This makes the process more undefined but leads to an extensive engagement until the end.