Sound Localization: New Research and Potential Applications

In Music, the Brain and Ecstasy, Jourdain (1997) spends several pages discussing sound localization (1997, 20–24), or the ability to distinguish where in space a particular sound is coming from. Jourdain claims that for modern humans “what sounds are” is far more important than “where they are” (1997, 20). Nevertheless, along with most animals, our inner ears, outer ears and brains have evolved in such a way that provides the ability to precisely locate a sound in space. Localization is one of the most important functions of human hearing, and “is the primary concern of the most primitive parts of the auditory brain” (Jourdain 1997, 20). In particular, sound localization draws upon multiple sources of information, engages key parts of the musical brain, and has the potential to be further enjoyed and enhanced using new technologies, including binaural and three-dimensional audio recordings and audio games.

            Sound localization depends on several kinds of environmental sound information. Interaural arrival time, or the difference in time by which a sound arrives at each ear, and the volume, or difference in intensity or loudness of sound, as it arrives at each ear together help determine the location of the sound on the horizontal plane, or in azimuth (Jourdain 1997, 21; Otte et al. 2013, 261). In addition, through experience with environmental sound, we are also able to use our knowledge of past sound experiences to determine the distance of the source of a sound (Jourdain 1997, 22). The mind uses these different kinds of information to calculate the location of the sound in both the horizontal and vertical planes.

            We also use these different kinds of sonic information for a closely related task—distinguishing one particular sound from among many, such as picking out the sound of just one person’s voice in a crowd of people talking, or a “cocktail party situation” (Zündorf, Lewald, and Karnath 2013). Recent research has identified the left auditory cortex, and in particular the left planum temporale (PT), as the primary regions of the brain invoked in spatial localization (Zündorf, Lewald, and Karnath 2013). This is interesting, in part, because the left PT is known to be associated with language, musical ability, and in particular perfect pitch (Schlaug et al. 1995). Though we know that the left PT is already somewhat more developed in utero than the right PT in general and is acutely more developed among professional musicians, particularly those with perfect pitch, it is still unclear to what extent musicians develop stronger left PTs or those with stronger left PTs are more likely to become musicians (Schlaug et al. 1995, 700).

            Implicitly, Jourdain’s discussion of localization assumes that most people have similar capacities for sound localization, and he fails to explore why some people are better at localization than others. For example, he explains that lower frequencies are easier to localize than others (Jourdain 1997, 21). Meanwhile, he also explains that as we age, our abilities to hear high frequency sounds progressively diminishes, which is also known as sensory presbycusis, (Jourdain 1997, 17; see also Dobreva, O’Neill, and Paige 2011, 2484). However, Jourdain never directly addresses whether localization ability changes with age. In contrast, recent work has documented that sound localization tends to become less accurate with age, even when experiments adjust for metabolic or flat presbycusis, which is diminished hearing at all frequencies (Dobreva, O’Neill, and Paige 2011). Interestingly, older adults also perform less well than younger adults, even for low-frequency sounds that have been adjusted to higher volumes (Dobreva, O’Neill, and Paige 2011, 2484). Similarly, recent research explores other correlates of the capacity for sound localization. For example, Zundorf et al. (2011) find that women overall are less accurate than men at identifying the location of a single sound alone or as one of many sounds. However, this gender difference is only significant when it comes to identifying the location of one sound in the presence of many sounds at once. The implications of these findings for music performance and appreciation are unclear but worthy of further investigation.

            When it comes to audio recordings, Jourdain laments that the sonic complexities of what-sounds-are-coming-from-where are lost as compared to the vividness when one is present at a live musical performance for at least two reasons. First, most recordings combine and condense the direct sounds of instruments with the sounds that are reverberating throughout the concert hall. Second, most people listen to recordings through inferior speakers in small rooms that reverberate in ways that do not match the original room or the initial sound design of the recordings (Jourdain 1997, 24). In contrast, Jourdain explains the “magical” effect of binaural recordings, which are made by placing a microphone in each of the ear canals of a life-sized dummy-head with typical pinnae (Jourdain 1997, 24). These kinds of recordings have been largely overlooked because their effects can only be appreciated by listening with headphones and are completely lost when heard on loudspeakers. But even though the “iPod Revolution” (The iPod Revolution 2007) has made ear buds and headphones ubiquitous, many binaural recordings remain underutilized and underappreciated.

            Binaural and three-dimensional sound are also used in to great effect in audio games, a recent genre of computer games that rely on aural rather than visual feedback for gameplay (see Targett and Fernström 2003; Friberg and Gärdenfors 2004; Röber and Masuch 2005). Originally developed for people with visual impairments, audio games offer two distinct advantages over video games. First, three-dimensional sound is easier to produce than three-dimensional video. And second, unlike video games, where increasingly smaller screens diminish the experience of gameplay, audio games can be enjoyed without the need for any screen at all, requiring only inexpensive headphones or ear buds to explore a rich, three-dimensional sonic landscape. This makes audio games ideal for mobile devices like tablets and smartphones. For example, the game Open Field Echo Sounder (Smolenski 2014) uses the Global Positioning System (GPS) capabilities of iPhones and Android devices to place virtual targets around a player standing in an open field. Sonic cues are given that direct the player toward the targets (Smolenski 2014). As we learn more about how we localize sound and as audio technology continues to improve, new opportunities may be realized for developing not only audio tools for play and enjoyment, but also for helping people with hearing loss re-train their localization abilities.

References

Dobreva, Marina S., William E. O’Neill, and Gary D. Paige. 2011. “Influence of Aging on Human Sound Localization.” Journal of Neurophysiology 105 (5): 2471–86. doi:10.1152/jn.00951.2010.

Friberg, Johnny, and Dan Gärdenfors. 2004. “Audio Games: New Perspectives on Game Audio.” In Proceedings of the 2004 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, 148–54. ACE ’04. New York, NY, USA: ACM. doi:10.1145/1067343.1067361.

Jourdain, Robert. 1997. Music, the Brain, and Ecstasy. New York: HarperCollins.

Otte, Rik J., Martijn J. H. Agterberg, Marc M. Van Wanrooij, Ad F. M. Snik, and A. John Van Opstal. 2013. “Age-Related Hearing Loss and Ear Morphology Affect Vertical but Not Horizontal Sound-Localization Performance.” Journal of the Association for Research in Otolaryngology 14 (2): 261–73. doi:10.1007/s10162-012-0367-7.

Röber, Niklas, and Maic Masuch. 2005. “Playing Audio-Only Games: A Compendium of Interacting with Virtual, Auditory Worlds.” In DiGRA 2005: Changing Views: Worlds in Play, 2005 International Conference. Vancouver, Canada. http://summit.sfu.ca/item/243.

Schlaug, G., L. Jancke, Y. Huang, and H. Steinmetz. 1995. “In Vivo Evidence of Structural Brain Asymmetry in Musicians.” Science 267 (5198): 699–701. doi:10.1126/science.7839149.

Smolenski, Bob. 2014. “Open Field Echo Sounder.” Medium. September 3. https://medium.com/@bobsmo/open-field-echo-sounder-game-app-377610ef64f0.

Targett, Sue, and Mikael Fernström. 2003. “Audio Games: Fun for All? All for Fun.” In International Conference on Auditory Display. Boston, MA, USA. http://dev.icad.org/websiteV2.0/Conferences/ICAD2003/paper/53%20Targett.pdf.

The iPod Revolution. 2007. Documentary. Discovery Channel.

Zündorf, Ida C., Hans-Otto Karnath, and Jörg Lewald. 2011. “Male Advantage in Sound Localization at Cocktail Parties.” Cortex 47 (6): 741–49. doi:10.1016/j.cortex.2010.08.002.

Zündorf, Ida C., Jörg Lewald, and Hans-Otto Karnath. 2013. “Neural Correlates of Sound Localization in Complex Acoustic Environments.” PLoS ONE 8 (5): e64259. doi:10.1371/journal.pone.0064259.