Journal of Sonic Studies, volume 6, nr. 1 (January 2014)Iain McGregor; Phil Turner; David Benyon: USING PARTICIPATORY VISUALISATION OF SOUNDSCAPES TO COMPARE DESIGNERS’ AND LISTENERS’ EXPERIENCES OF SOUND DESIGNS
1.4 Visualising Soundscapes
We are, of course, not the first to propose visualizing sound and its attributes. The painter Wassily Kandinsky translated atonal music into canvases (Brougher and Zilczer 2005). Another artist, Roy de Maistre curated in 1919 the exhibition Colour in Art where the musical notes A to F were converted to colours (A = red), paintings were accompanied by music, and colour charts were made available for the audience (Alderton 2011). Gibson (Gibson 2005) displayed different frequencies using colour, as did Matthews, Fong and Mankoff (Matthews et al. 2005). Circles have been regularly used to represent sound and can be found in a variety of visualisation schemes (Azar, Saleh and Al-Alaoui 2007; Frecon, Stahl, Soderberg and Wallberg 2004; Helyer, Woo and Veronesi 2009). Servigne, Kang and Laurini (Servigne et al. 2000) proposed that the varying intensities of noise could be visualized by altering the radius of circles. Gibson also adopted this approach by indicating the volume of a sound in a mix by the object’s size, with louder being larger than quieter. Shape has been used to designate the articulation of musical notes: legato = rounded, staccato = polygon (Friberg 2004).
Abstract shapes have also been applied to visualize phonemes that are not recognised by a phoneme recognition system, with high frequency sounds having spiky irregular forms (Levin and Lieberman 2004). Opacity has been used to a limited extent to communicate the volume or loudness of a sound event (Mathur 2009; Radojevic and Turner 2002; Thalmann and Mazzola 2008). Servigne, Laurini, Kang and Li (Servigne et al. 1999) suggested that graphic semiology would be appropriate for displaying sounds, proposing that smiling faces overlaid onto a map could be used to display a participant’s preferences, a smile represented “nice”, a neutral expression “neutral”, and a frown “not so good”. Bertin’s 1967 theory of cartographic communication was used to create the visualisation. Bertin proposed that the visual variables of shape, size, value, orientation, hue, texture, x and y coordinates could be applied to point, line, and area symbols. Monmonier (Monmonier 1993) argued that Bertin’s variables were also suitable for text, which could act as symbols within visualisations.
Figure 1 holds the set of symbols we have developed iteratively based on the literature highlighted above to represent the components of a soundscape (McGregor, Crerar, Benyon and Leplatre 2008; McGregor, Leplatre, Turner and Flint 2010). Each attribute of a classified sound event is visualised according to the symbols below, and then placed on a grid according to its perceived spatial location.