Reference: Thompson, W. F., Graham, P., and Russo, F. A. (2005) “Seeing music performance: Visual influences on perception and experience”, Semiotica 156: 177-201.
This long and detailed article addresses a very important and interesting aspect of music performance-its visual dimensions. It discusses the extent to which these aspects contribute to the communication between performer and a listener. The authors maintain that there are several levels at which facial expressions and gestures might affect our perception of music. First, there is basic level at which listeners’ attention is directed to critical acoustic information at a certain moment of time. The authors argue that such cues can increase musical intelligibility. They draw an analogy referring to the face of a speaker, that when seen may increase the intelligibility of speech in a noisy environment. Then there is a perceptual level, at which visual cues are given to the listener to indicate important melodic, harmonic, and rhythmic events. The authors suggest that at this level, facial expressions and gestures are intentionally introduced by performers as a way of sharing with listeners their understanding of the significance of such events, as well as emphasizing the musical activity as a shared experience between performer and listener. Finally, the authors observe that visual information is “highly effective at conveying persona and attitude” (p. 204), arguing that facial expressions and other bodily gestures help performer to personalize the music, creating the feeling of reciprocal human interaction during the performance. Next, the authors discuss a variety of different media technologies, which can be used to experience music performance, and argue that each technological medium provides listener with different experience. They put forward a concept of music as “multimedially performed and multimodally experienced” (p. 205), and present empirical findings that appear to support this model. Two case studies are presented in order to discuss the relationship between visual and aural aspects of performance. The authors thoroughly examine the use of facial expression, body movement, and gesture in two filmed performances, one of B. B. King (playing Blues Boys Tune), and another by Judy Garland (singing Just in Time). Interestingly, the analysis of these samples is based on structured interviews by a trained musicologist of two other musicologists. The authors observe that the wide range of potential effects that visual information can have on our perception of music suggests the necessity of further and more systematic research in this area. Accordingly, they designed five experiments to assess visual influences on two types of musical judgments. In three experiments, they investigated visual influences on structural interpretations of music and in two experiments - visual influences on emotional interpretations of music. The experiments confirmed that visual characteristics of performance influence music experience at a perceptual and an emotional level. In conclusion, the authors observe that, during performance, listeners incorporate visual and aural aspects to form an integrated audio-visual mental representation of music.
In my opinion, visual dimension is one of the most interesting and significant characteristics of music performance. In this regard, the article is definitely thought provoking. The authors suggest that these aspects are greatly overlooked by most psychological research, which consider them as unessential to the music. This may be true concerning systematic scientific investigation; however, it is extremely rare that visual aspects of performance are disregarded by a common listener (especially by those with limited musical training). People are extremely attentive to these details; they watch musicians closely, and, at times, it appears that these aspects are, perhaps, of even greater importance for them than the music itself. When Evgeny Kissin toured in Israel, my mother-in-law (who is a professional pianist, performer and teacher with lots of experience) attended one of his performances with a friend of hers (who has no formal training in music). After the concert, my mother-in-law, agitated and extremely excited, conducted a thorough analysis of the performance of this, perhaps one of the greatest pianists of all time, focusing in particular on his interpretation, phrasing, dynamics and other related specific details. The friend listened attentively and then expressed her curiosity regarding the reason behind my mother-in-law decision to buy more expensive tickets in the second row. My mother-in-law replied that she wanted to see the pianist’s hands, after which the friend thoughtfully observed, “But what ugly faces and dreadful grimaces he makes all the time!” I think this observation is not uncommon. Moreover, it is very characteristic. The authors of the article argue that listeners form an integrated audio-visual image of music. If this were correct, what kind of mental representation of the music the aforementioned friend would have? Will the music that she just listened to be always associated for her with the “ugly” and contorted countenance of the performer?
Then, perhaps, this could be further subdivided into two related questions. Do the twisted features or body movements of performer bear any interpretational connotations? Do these movements constitute an intentional message of a performer to a listener? Does it convey or elucidate musical meaning? Or, it’s just an involuntary muscle movement, correlated with the level of difficulty of a certain passage, or associated with the intensity of performers’ emotional involvement? And, another question, how are these aspects perceived by listener. Do listeners, indeed, combine visual and audio aspects of performance to create an integrated image of music, or visual characteristics simply distract them from the music? It appears that the latter could be true.
Obviously, there is a great disparity between professional musician’s approach to a listening process and that of a person with no formal music training. While musicians would typically focus on the specific music qualities of performance, disregarding other attributes, non-musicians would be greatly amused and distracted by other characteristics, including visual aspects of performance. This is especially the case with teenagers and young kids. When I attempt to discuss with them the music performances they attend, it appears that visual aspects of performance make more profound and enduring impression on them, than any specific musical characteristics.
There is also another issue, which I think worth mentioning in this context. It seems that visual aspects of performance, including facial expressions, gestures, and body movements are culturally determined. For instance, I know that in Russia, such movements were not encouraged. It was thought that a musical message should be transmitted using specific musical means of expression, such as dynamics, phrasing, tempos, articulation, and so on, rather than by means of the excessive and disproportionate visual indications.
In short, this is an interesting subject to discuss, and I think it deserves to be further investigated.
Here is a couple of links to video recordings of the music performances. This is amazing how differently these artists express themselves musically and visually. Is it culture? Differences in professional training? Fashion?
David Oistrakh plays Tchaikovsky Violin Concerto
Jascha Heifetz plays the same Concerto
Itzhak Perlman plays Tchaikovsky Violin Concerto
Evgeny Kissin plays Tchaikovsky Piano Concerto
Van Cliburn plays the same Concerto